JUNOS RIB-GROUPS (1/2)

During the JNCIE-SP preparation session we had a few weeks ago, I mentioned that if someone had any suggestions about topics for my blog, I would be happy to consider them. Someone answered: rib-groups! And I thought: “oh yeah, that’s a good one!”

Rib-groups seem so difficult at first, but once you understand them, you realize that they are actually not difficult at all, and actually are pretty cool.  

In all honestly, I was lucky to sit in a class with Shawn Coville, one of the best instructors I’ve ever had, during my early Juniper years. In that class, he explained rib-groups using a very simple analogy that made so much sense and stuck with me.

Ever since, I have been explaining rib-groups to people using the same analogy, though with my own personal twists, and extras.  

The story goes like this:

Imagine that you have a water faucet in your backyard. Normally, you go open the water, and place a bucket on the ground to collect the water, that you use to water some of your plants. Quite simple!

One day you decide to place a Y-shaped water pipe under the faucet. I am sure it is pretty obvious to you that water comes in on the top, and comes out of the two ends at the bottom.

You then place a couple of buckets strategically located to collect the water coming out of the two ends of the pipe.  You are filling out two buckets instead of one. yeah!

OK, I know! Just hang in there!

Now, you notice that there are some pebbles mixed with the water, and let’s say that for whatever reason you want to collect clean water only on bucket #2.

Well, that’s pretty simple too. You just need some kind of filter (yeah, a pasta drainer, like the one in my picture might actually work pretty well). You place it right at the end of the pipe, and there you go, no more pebbles in bucket #2.

At this point you might be losing your patience, LOL! (hopefully not) and thinking: “What does this have anything to do with rib-groups?”.

Let me explain.

The basics:

Replace the water with routes, the pipe with a rib-group, the buckets with routing tables, the faucet with routing protocols, and the filter with a policy, and you get this:

Routes come out of the protocols, and get installed in the routing table(s).

Now, by default, if you configure an IPv4 address for an interface in the master or default routing instance (not part of a routing instance you create), the router installs a local /32 route, and a direct /n (interface routes), in inet.0 (the default IPv4 routing table). There are actually some simple rib-groups be default, that tell the router go to inet.0 for example.

Here is an example:

We are configuring the IPv4 address 172.16.1.1/24 under interface fe-0/0/0.0. The router creates a /32 local address, which is the route to the interface itself. It also creates a /24 direct route that represents the network connected to interface fe-0/0/0.0.

Now the most important detail here is that these two routes are places in inet.0.

Also by default, when you configure a routing protocol (e.g. OSPF) in the main routing instance, the protocol installs the routes into that same default IPv4 routing table inet.0.  

Here you can see we that we running OSPF between R1 and R2, and R1 is learning about the lo0.0 interface address from R2. Once again, because OSPF is configured under the main routing instance the OSPF routes are installed in inet.0

So far we know that any IPv4 interface, OSPF, BGP, ISIS, static, and so on route, not configured under a routing instance, is installed in inet.0 by default. What happens if we do create a routing instance, and configure interfaces or protocols under that routing instance?

In the next example, we are creating a routing instance (virtual router) named VR1, adding interfaces lo0.1 and fe-0/0/0.0 , and configuring OSPF.

The interface routes (local and direct) associated with lo0.1 and fe-0/0/0.0, as well as any OSPF route learned from neighbors on fe-0/0/0.0, will by default go to VR1.inet.0.

Now, before we move any further, let’s also talk about import vs. export.  We have seen examples of routes being installed (imported) into a routing table.  Like collecting the water in the buckets. 

But there is a second component to the history: taking routes (exporting) from the routing table and sending that information to neighbors.

Which in our silly example would be like collecting the water and then watering the plants.

Most of what we are going to discuss in the rest of this article will be about importing routes, but it is important that you understand the idea of import vs. export. When you start going deeper into routing policies (will probably cover that in a different article) understanding this idea is critical essential to being able to configure things like redistribution properly.

Routes for interfaces and protocols under the main routing instances, are placed in inet.0, but also, protocols under the main routing instances export information from inet.0  

As an example, let’s say we create static routes in both inet.0 and VR1.inet.0, and then configure an export policy to redistribute the static routes into BGP.

Notice that the policy does not define a specific static route or routing table to match on. Would the static routes from both inet.0 and VR1.inet.0 be advertised?

The answer is NO! The only static route that will be considered for export is the route installed in inet.0, because BGP is configured under the main routing instance.

If we changed the topology to this:

Without changing the policy at all, this is what we would have in R2’s routing table:

The static route configured under the routing instance on R1 was advertised this time, because now BGP is configured under the routing instance.

Let’s now learn how to change the default behavior and learn how to install routes into a different routing table, or even multiple routing tables at the same time.

Using rib-groups

There are several situations that require routes to be installed in multiple routing tables, but for now try not to ask: “why would I want to do this?” and focus on how the rib-group works and the steps. I will discuss some real use cases in the second part of this article.

So, let’s go back to the idea of having two (or more) buckets to collect water, though now you know that we are actually using routing tables to collect routes.

What we are doing now is this: rather than allowing the router to install routes in inet.0, for example, as it would do by default, we tell it to “install” the routes into a rib-group, which basically translates into: install the routes into this other routing table(s).

You can see that in the picture below:

We are telling OSPF that rather than installing the routes into inet.0 we want the routes into both inet.0 and inet.2.

Using our analogy again it would be something like this:

We tell the router to install the routes into a rib group using:

We define which routing tables the routes will be installed into, when we create the rib-group under routing-options using:

The import-rib command defines the routing-tables where the routes are actually going to be installed.  (creates the buckets to collect the routes)

NOTE: you need to make sure you configure the routing table the routes would go to by default first. Else you might get an error message when trying to commit. We will talk more about this later.

Let’s look at this example now:

We have two vMX1 routers connected via ge-0/0/0.0 and we want to run OSPF, and on vMX1 we want the route for the loopback interface of vMX2, to be installed in both inet.0 and VR1.inet.0.

Notice that interface ge-0/0/0.0 is part the main routing instance.

We first configure the interfaces, and create the routing instance on vMX1, and also configure the interfaces on vMX2.

Then we check the contents of the routing table on both routers. 

You can see that in both routers a /32 direct route was created for lo0.0 interface, in inet.0 and a /32 direct route was created for the lo0.1 interface, in VR1.inet.0 on vMX1.

Also, a /32 local route and a /24 direct route were created for interface ge-0/0/0.0. Because ge-0/0/0.0 belongs to the master routing-instance, these routes were installed in inet.0, and no route was created in VR1.inet.0. 

We now configure OSPF between vMX1 and vMX2 and make sure that vMX1 has learned the address of the loopback interface of vMX2.

Because OSPF is configured under the master (or default) routing-instance, any route learned via OSPF will be installed in inet.0 by default.

We validate that the route for 192.168.2.2/32 is present in inet.0 on vMX1.

Let’s now create the rib-group that will allow us to place this route also under VR1.inet.0, and apply it to OSPF.

We check the contents of VR1.inet.0 and voila! Now there is a route for the loopback interface of vMX2 there!

And just to make sure, let’s also check the contents of inet.0:

Yeap! The route is also there. Mission accomplished!!! OR is it really?

Well, only partially! We did get the route to be installed in VR1.inet.0 and yes, the route is even active, but does the router know how to get to 10.1.1.2 from VR1.inet.0?

Nope! If we tried to ping 192.168.2.2 from the routing-instance for example, the ping would fail.

Then, how come was the route installed in VR1.inet0?

Let me clarify one detail first: in my diagrams, I like to represent the interfaces as connected to the routing tables, to show you that the interface (local and direct) routes for that particular interface will be installed in that particular routing table, and traffic that arrives at the interface will be forwarded based on information on that routing table.

That does NOT mean that routing information arriving at the interface from say OSPF, goes directly to the routing table. Routing information goes to the Routing Engine which determines which route will be installed in the routing table, and which route will become active.

Also, remember that OSPF is a Link State routing protocol, which means it does not advertise routes but LSAs (Link State Advertisements). These LSAs are placed in the Link State Database, and SPF (Shortest Path First) calculates the OSPF routes that are then installed in the routing table.

In our example, without a rib-group the OSPF routes calculated by SPF go straight to inet.0, since OSPF is configured on the main routing instance.

Once we add the rib-group, the routes calculated by SPF are “installed” in the rib-group. In other words, the routes are installed in BOTH inet.0 and VR1.inet.0.

And again, the fact that we now have this active OSPF route in VR1.inet.0 for 192.168.2.2/32 does not mean we can now reach 192.168.2.2 from within the VR1 routing-instance. The next-hop 10.1.1.2 is not reachable from within the routing instance, and we would need to also add the interface routes for fe-0/0/0.0 to VR1.inet.0, to make 192.168.2.2 reachable. We will see how to achieve this as part of the next example.

Let’s now say that we create a static route to reach 192.168.2.2/32 using 10.1.1.2 as next-hop, in both inet.0 and VR1.inet.0 and stop using OSPF for a moment.

We create the static route directly under routing-options for the master routing instance, and a static route under routing-options under the routing-instance VR1:

We check the routing tables:

And find that though the static route was installed under inet.0, it is not under VR1.inet.0 even though we configured a static route under the routing instance as well.

Why is that?

For a static route to be valid, the next-hop has to be reachable, and it has to be reachable from the routing instance where the static route is to be created on. 

In other words, if I am trying to create a static route in inet.0, I need to have a route to reach the static route’s next-hop in inet.0. In the same way, if I am trying to create a static route in VR1.inet.0, I need to have a route to reach the static route’s next-hop in VR1.inet.0.

Under the main routing instance, this requirement is met:

But under VR1.inet.0 there is no route for 10.1.1.2, so the next-hop is not reachable:

Remember that ge-0/0/0.0 belongs to the master routing instance, and therefore the direct and local routes associated with that interface are placed in inet.0.

The only interface configured under the VR1 routing instance is lo0.1. Thus, the only direct route in VR1.inet.0 will come from lo0.1  

We need to place the routes associated with ge-0/0/0 in both the master routing instance and the VR1 routing instance.

Trying to configure ge-0/0/0.0 under VR1, would just swap the problem.

And one more thing: an interface can only belong to a single routing instance. If after moving the interface inside the routing instance we attempted to use it under the master routing instance, we would get a commit error:

So, how do we make it possible for the two static routes in the two instances (master and VR1), both using 10.1.1.2 as next hop, to be valid at the same time?

That’s right! Let’s use a rib-group. In fact, we can use the one we already created for our OSPF testing, though instead of applying it to OSPF, we are going to apply it to interface routes (direct and local routes).

Let’s check the results:

If we compare the tables, we find some interesting pieces of information:

1) The management interface fxp0 is NOT affected by the rib-group configuration.

The management interface can only be part of the master routing instance OR the management routing-instance if you configure it.

2) The direct route for ge-0/0/0.0, and ALSO the direct route for the loopback interface of the master routing instance (lo0.0) have been installed in both routing tables. The loopback interface under the VR1 routing-instance was not installed in inet.0.

This is because we applied the rib-group under the main routing instance:

set routing-options interface-routes

and not under the VR1 routing instance:

set routing-instance VR1 routing-options interface-routes.

When we use set routing-options interface-routes we are telling the router that direct and local routes associated with interfaces in the main routing instance (going to inet.0 by default) need to be processed by the rib-group, instead of being installed directly into inet.0

When we use set routing-instance VR1 routing-options interface-routes we are telling the router that direct and local routes associated with interfaces in the VR1 routing instance (going to VR1.inet.0 by default) need to be processed by the rib-group, instead of being installed directly into in VR1.inet.0 .

3) All the interfaces that belong to the master routing-instance now and in the future, except for fxp0, will be installed in both inet.0 and VR1.inet.0 with our current configuration.

In fact, if we added more interfaces to the master instance right now, they would immediately be placed in both inet.0 and VR1.inet.0.

What if I only want to share some interfaces?  In our example, we only need to share interface ge-0/0/0.0.

Remember the water filter?  That’s what we are going to use now.

Using a policy with rib-groups

You can use a routing-policy to control which routes are installed in which routing table, when using a rib-group.  

This requires two steps:

1. Create a routing policy, specifying which route are allowed into each routing table
2. Create a rib-group (as previously described) and reference the policy.

For the first step, we create the policy under policy-options using:  

Notice the “to” statement.  This statement is the key of how this policy will help us, as it defines which routing table the route are going to.

The combination of the from statement(s) and the to statement define exactly which route(s) will be installed on each routing table.

For example, if we configure the following two terms within a policy:

Term 1 defines that prefix 10.1.1.1/32 can be installed in inet.0, while term 2 defines that the same prefix cannot be installed in inet.2.

For the second step, we apply the policy to the rib-group using:

set rib-groups <rib-group name> import-policy <policy_name>

Let’s apply this concept to our example in the previous section.

We wanted to add a static route to reach the loopback interface of vMX2 from both inet.0 and VR1.inet.0 (routing instance).

We found that in order for the static route to become active, the next-hop had to be reachable, which is easy, when we are trying to add the static route to inet.0, because ge-0/0/0.0 is part of inet.0. But for the same reason, the next-hop was not reachable from VR1.inet.0.

We solved the problem by sharing the interface routes and placing them in both inet.0 and VR1.inet.0 using a rib-group: 

But we ended up having ALL interface routes in both inet.0 and VR1.inet.0.

In our exercise this might not be too bad, but imagine a large router, with hundreds of interfaces, and hundreds of routing instances, and then you share all interface routes, without any filtering. 

Routers support a certain number of routes that you don’t want to waste installing unnecessary routes in multiple routing table.  Again, think of a large core router that might be already holding thousands of routes.

So, what we really want to do is install the interface routes of ge-0/0/0.0 ONLY . More precisely, we ONLY need the direct route of ge-0/0/0.0.

So here is what we are going to do:

1) Create a policy that allows the direct route for interface ge-0/0/0.0 to be installed in VR1.inet.0 and rejects any other route going to the same routing table.

2) Apply the policy to the rib-group:

3) After applying the changes, we check the routing tables again:

We find that nothing has changed for inet.0. Our policy does not touch inet.0, and therefore the default behavior applies.

VR1.inet.0 now shows exactly what we wanted: Besides the direct route for lo0.1 (which belongs to the routing instance) the only direct route we have in the table is the one associated with interface ge-0/0/0.0.

This route allows the next-hop of our static route to be reachable, thus allowing the static route to become active.

Pretty AWESOME! 

Now, would this work the same way if we moved the interface ge-0/0/0.0 into the routing instance? Can I also install the direct route in both tables if the interface belongs to the routing-instance?

Sure! BUT: you need to consider that now the direct route goes to VR1.inet.0 by default, and that you are applying the rib-groups to an interface WITHIN the routing instance. 

Let’s say again that we add interface ge-0/0/0.0 to the routing-instance:

We check the routing tables afterwards, and we find that the direct and local routes for ge-0/0/0 are not part of VR1.inet.0 (NOT part of inet.0) and the static route in inet.0 for the loopback of vMX2 is gone:  

We want to place these direct routes for interface ge-0/0/0.0 into inet.0 as well, so that our static route becomes valid.

Remember that configuring this:

applies to interface that belong to the master routing instance (inet.0).

The interface that we are now concerned about does not belong to inet.0.  We need to go under the routing instance and configure the interface-routes for the interfaces that belong to that routing instance.

We apply the new configuration and we find that interface ge-0/0/0.0 was installed in inet.0 and VR1.inet.0, but so was the direct route for loopback and also the local route of ge-0/0/0.0

Why was the direct route for interface lo0.1 also installed in inet.0?

If we review the policy that currently is applied to the rib-group we will quickly find the answer:

This policy controls the route to be installed in VR1.inet.0, which we are not even trying to do right now, and does not touch the routes going to inet.0.

So, let’s modify the policy:

You can explicitly say to rib inet.0 within terms 3 and 4 but it is not needed in this case.

So far, so good!   But let’s forget about the static routes and bring up OSPF again.

ORDER MATTERS!

We know want the OSPF route that is being installed in VR1.inet.0 to be installed in both inet.0 and VR1.inet.0 instead.

The direct route for ge-0/0/0 is already on both routing tables with the current configuration, and the OSPF route is installed in VR1.inet.0 What we need is to apply a rib-group to OSPF as well.

oops! Commit fails!!!!

Due to some implementation internals, that I haven’t fully understand yet, and that I kind of decided to ignore some time ago (and has not affected my ability to use rib-groups like a pro), the router requires that the routing table the protocols would install the routes in by default be listed first within the rib-group.

That means that because OSPF in this case is configured under the VR1 routing instance, its routes would by default be installed in VR1.inet.0.  Thus, VR1.inet.0 should be listed first.

Swapping the two routing tables will do the trick:

We also need to again fix our policy:

And just as we wanted, the OSPF route is now in inet.0 and VR1.inet.0:

All right! I think you got enough information to now be dangerous and have fun with rib-groups for a little bit!

I am going to close this article with a quick summary of the steps, with the assurance that I will writing the follow up article with some real rib-groups use cases soon.

CONFIGURATION STEPS SUMMARY

1) Define rib-group under the routing-options hierarchy level

The import-rib defines which tables routes should be installed in

2) Apply the rib-group to routing protocols, interface routes, or both, as needed

3) Create a routing-policy if needed

4) Apply the policy to the rib-group

The import-policy controls which routes are installed in each routing table.

Routing